Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use
Identifieur interne : 000A18 ( Main/Exploration ); précédent : 000A17; suivant : 000A19Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use
Auteurs : Karen Fort [France] ; Gilles Adda [France] ; Benoît Sagot [France] ; Joseph Mariani [France] ; Alain Couillault [France]Source :
Descripteurs français
- Wicri :
- topic : éthique.
English descriptors
Abstract
This article is a position paper about Amazon Mechanical Turk, the use of which has been steadily growing in language processing in the past few years. According to the mainstream opinion expressed in articles of the domain, this type of on-line working platforms allows to develop quickly all sorts of quality language resources, at a very low price, by people doing that as a hobby. We shall demonstrate here that the situation is far from being that ideal. Our goal here is manifold: 1- to inform researchers, so that they can make their own choices, 2- to develop alternatives with the help of funding agencies and scientific associations, 3- to propose practical and organizational solutions in order to improve language resources development, while limiting the risks of ethical and legal issues without letting go price or quality, 4- to introduce an Ethics and Big Data Charter for the documentation of language resource
Url:
DOI: 10.1007/978-3-319-08958-4_25
Affiliations:
- France
- Grand Est, Lorraine (région), Nouvelle-Aquitaine, Poitou-Charentes
- La Rochelle, Metz, Nancy
- Université de La Rochelle, Université de Lorraine
Links toward previous steps (curation, corpus...)
- to stream Hal, to step Corpus: 001A01
- to stream Hal, to step Curation: 001A01
- to stream Hal, to step Checkpoint: 000955
- to stream Main, to step Merge: 000A20
- to stream Main, to step Curation: 000A18
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use</title><author><name sortKey="Fort, Karen" sort="Fort, Karen" uniqKey="Fort K" first="Karen" last="Fort">Karen Fort</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-150772" status="VALID"><idno type="RNSR">201120979K</idno><orgName>Semantic Analysis of Natural Language</orgName><orgName type="acronym">SEMAGRAMME</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/semagramme</ref></desc><listRelation><relation active="#struct-129671" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-423086" type="direct"></relation><relation active="#struct-206040" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-129671" type="direct"><org type="laboratory" xml:id="struct-129671" status="VALID"><idno type="RNSR">198618246Y</idno><orgName>INRIA Nancy - Grand Est</orgName><desc><address><addrLine>615 rue du Jardin Botanique 54600 Villers-lès-Nancy</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/nancy</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-423086" type="direct"><org type="department" xml:id="struct-423086" status="VALID"><orgName>Department of Natural Language Processing & Knowledge Discovery</orgName><orgName type="acronym">LORIA - NLPKD</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.loria.fr/la-recherche-en/departements/Knowledge-and-Language-Management</ref></desc><listRelation><relation active="#struct-206040" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation></org></tutelle><tutelle active="#struct-206040" type="indirect"><org type="laboratory" xml:id="struct-206040" status="VALID"><idno type="IdRef">067077927</idno><idno type="RNSR">198912571S</idno><idno type="IdUnivLorraine">[UL]RSI--</idno><orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName><orgName type="acronym">LORIA</orgName><date type="start">2012-01-01</date><desc><address><addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.loria.fr</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation><relation active="#struct-413289" type="direct"></relation><relation name="UMR7503" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-413289" type="indirect"><org type="institution" xml:id="struct-413289" status="VALID"><idno type="IdRef">157040569</idno><idno type="IdUnivLorraine">[UL]100--</idno><orgName>Université de Lorraine</orgName><orgName type="acronym">UL</orgName><date type="start">2012-01-01</date><desc><address><addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lorraine.fr/</ref></desc></org></tutelle><tutelle name="UMR7503" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Nancy</settlement><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Adda, Gilles" sort="Adda, Gilles" uniqKey="Adda G" first="Gilles" last="Adda">Gilles Adda</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-202" status="OLD"><orgName>Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur [Orsay]</orgName><orgName type="acronym">LIMSI</orgName><desc><address><addrLine>Université Paris Sud (Paris XI) Bât. 508 BP 133 91403 ORSAY CEDEX</addrLine><country key="FR"></country></address><ref type="url">http://www.limsi.fr/</ref></desc><listRelation><relation name="UPR3251" active="#struct-441569" type="direct"></relation><relation active="#struct-92966" type="direct"></relation><relation active="#struct-93591" type="direct"></relation></listRelation><tutelles><tutelle name="UPR3251" active="#struct-441569" type="direct"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-92966" type="direct"><org type="institution" xml:id="struct-92966" status="VALID"><orgName>Université Paris-Sud - Paris 11</orgName><orgName type="acronym">UP11</orgName><desc><address><addrLine>Bâtiment 300 - 91405 Orsay cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.u-psud.fr/</ref></desc></org></tutelle><tutelle active="#struct-93591" type="direct"><org type="institution" xml:id="struct-93591" status="VALID"><orgName>Université Pierre et Marie Curie - Paris 6</orgName><orgName type="acronym">UPMC</orgName><desc><address><addrLine>4 place Jussieu - 75005 Paris</addrLine><country key="FR"></country></address><ref type="url">http://www.upmc.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Sagot, Benoit" sort="Sagot, Benoit" uniqKey="Sagot B" first="Benoît" last="Sagot">Benoît Sagot</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-54505" status="OLD"><idno type="RNSR">200818336A</idno><orgName>Analyse Linguistique Profonde à Grande Echelle ; Large-scale deep linguistic processing</orgName><orgName type="acronym">ALPAGE</orgName><date type="end">2016-01-31</date><desc><address><addrLine>Université Paris Diderot, Bât. Olympe de Gouges, case postale 7003, 75205 Paris cedex 13 - INRIA Rocquencour</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/alpage</ref></desc><listRelation><relation active="#struct-86790" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-300301" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-86790" type="direct"><org type="laboratory" xml:id="struct-86790" status="VALID"><idno type="RNSR">196718247G</idno><orgName>INRIA Paris-Rocquencourt</orgName><desc><address><addrLine>INRIA Rocquencourt : Domaine de Voluceau, Rocquencourt B.P. 105 78153 le Chesnay Cedex / INRIA Paris - 23 avenue d'Italie 75013 Paris</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/centre/paris-rocquencourt</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-300301" type="direct"><org type="institution" xml:id="struct-300301" status="VALID"><orgName>Université Paris Diderot - Paris 7</orgName><orgName type="acronym">UP7</orgName><desc><address><addrLine>5 rue Thomas-Mann - 75205 Paris cedex 13</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-paris-diderot.fr</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Mariani, Joseph" sort="Mariani, Joseph" uniqKey="Mariani J" first="Joseph" last="Mariani">Joseph Mariani</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-202" status="OLD"><orgName>Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur [Orsay]</orgName><orgName type="acronym">LIMSI</orgName><desc><address><addrLine>Université Paris Sud (Paris XI) Bât. 508 BP 133 91403 ORSAY CEDEX</addrLine><country key="FR"></country></address><ref type="url">http://www.limsi.fr/</ref></desc><listRelation><relation name="UPR3251" active="#struct-441569" type="direct"></relation><relation active="#struct-92966" type="direct"></relation><relation active="#struct-93591" type="direct"></relation></listRelation><tutelles><tutelle name="UPR3251" active="#struct-441569" type="direct"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-92966" type="direct"><org type="institution" xml:id="struct-92966" status="VALID"><orgName>Université Paris-Sud - Paris 11</orgName><orgName type="acronym">UP11</orgName><desc><address><addrLine>Bâtiment 300 - 91405 Orsay cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.u-psud.fr/</ref></desc></org></tutelle><tutelle active="#struct-93591" type="direct"><org type="institution" xml:id="struct-93591" status="VALID"><orgName>Université Pierre et Marie Curie - Paris 6</orgName><orgName type="acronym">UPMC</orgName><desc><address><addrLine>4 place Jussieu - 75005 Paris</addrLine><country key="FR"></country></address><ref type="url">http://www.upmc.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Couillault, Alain" sort="Couillault, Alain" uniqKey="Couillault A" first="Alain" last="Couillault">Alain Couillault</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-40831" status="VALID"><orgName>Laboratoire Informatique, Image et Interaction</orgName><orgName type="acronym">L3I</orgName><desc><address><addrLine>Bâtiment Pascal Avenue Michel Crépeau F-17042 La Rochelle Cedex 1</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lr.fr/l3i</ref></desc><listRelation><relation name="EA2118" active="#struct-300311" type="direct"></relation></listRelation><tutelles><tutelle name="EA2118" active="#struct-300311" type="direct"><org type="institution" xml:id="struct-300311" status="VALID"><orgName>Université de La Rochelle</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">La Rochelle</settlement><region type="region" nuts="2">Nouvelle-Aquitaine</region><region type="old region" nuts="2">Poitou-Charentes</region></placeName><orgName type="university">Université de La Rochelle</orgName></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">HAL</idno><idno type="RBID">Hal:hal-01053047</idno><idno type="halId">hal-01053047</idno><idno type="halUri">https://hal.inria.fr/hal-01053047</idno><idno type="url">https://hal.inria.fr/hal-01053047</idno><idno type="doi">10.1007/978-3-319-08958-4_25</idno><date when="2014-07-25">2014-07-25</date><idno type="wicri:Area/Hal/Corpus">001A01</idno><idno type="wicri:Area/Hal/Curation">001A01</idno><idno type="wicri:Area/Hal/Checkpoint">000955</idno><idno type="wicri:explorRef" wicri:stream="Hal" wicri:step="Checkpoint">000955</idno><idno type="wicri:Area/Main/Merge">000A20</idno><idno type="wicri:Area/Main/Curation">000A18</idno><idno type="wicri:Area/Main/Exploration">000A18</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use</title><author><name sortKey="Fort, Karen" sort="Fort, Karen" uniqKey="Fort K" first="Karen" last="Fort">Karen Fort</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-150772" status="VALID"><idno type="RNSR">201120979K</idno><orgName>Semantic Analysis of Natural Language</orgName><orgName type="acronym">SEMAGRAMME</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/semagramme</ref></desc><listRelation><relation active="#struct-129671" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-423086" type="direct"></relation><relation active="#struct-206040" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-129671" type="direct"><org type="laboratory" xml:id="struct-129671" status="VALID"><idno type="RNSR">198618246Y</idno><orgName>INRIA Nancy - Grand Est</orgName><desc><address><addrLine>615 rue du Jardin Botanique 54600 Villers-lès-Nancy</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/nancy</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-423086" type="direct"><org type="department" xml:id="struct-423086" status="VALID"><orgName>Department of Natural Language Processing & Knowledge Discovery</orgName><orgName type="acronym">LORIA - NLPKD</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.loria.fr/la-recherche-en/departements/Knowledge-and-Language-Management</ref></desc><listRelation><relation active="#struct-206040" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation></org></tutelle><tutelle active="#struct-206040" type="indirect"><org type="laboratory" xml:id="struct-206040" status="VALID"><idno type="IdRef">067077927</idno><idno type="RNSR">198912571S</idno><idno type="IdUnivLorraine">[UL]RSI--</idno><orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName><orgName type="acronym">LORIA</orgName><date type="start">2012-01-01</date><desc><address><addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.loria.fr</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation><relation active="#struct-413289" type="direct"></relation><relation name="UMR7503" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-413289" type="indirect"><org type="institution" xml:id="struct-413289" status="VALID"><idno type="IdRef">157040569</idno><idno type="IdUnivLorraine">[UL]100--</idno><orgName>Université de Lorraine</orgName><orgName type="acronym">UL</orgName><date type="start">2012-01-01</date><desc><address><addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lorraine.fr/</ref></desc></org></tutelle><tutelle name="UMR7503" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Nancy</settlement><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Adda, Gilles" sort="Adda, Gilles" uniqKey="Adda G" first="Gilles" last="Adda">Gilles Adda</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-202" status="OLD"><orgName>Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur [Orsay]</orgName><orgName type="acronym">LIMSI</orgName><desc><address><addrLine>Université Paris Sud (Paris XI) Bât. 508 BP 133 91403 ORSAY CEDEX</addrLine><country key="FR"></country></address><ref type="url">http://www.limsi.fr/</ref></desc><listRelation><relation name="UPR3251" active="#struct-441569" type="direct"></relation><relation active="#struct-92966" type="direct"></relation><relation active="#struct-93591" type="direct"></relation></listRelation><tutelles><tutelle name="UPR3251" active="#struct-441569" type="direct"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-92966" type="direct"><org type="institution" xml:id="struct-92966" status="VALID"><orgName>Université Paris-Sud - Paris 11</orgName><orgName type="acronym">UP11</orgName><desc><address><addrLine>Bâtiment 300 - 91405 Orsay cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.u-psud.fr/</ref></desc></org></tutelle><tutelle active="#struct-93591" type="direct"><org type="institution" xml:id="struct-93591" status="VALID"><orgName>Université Pierre et Marie Curie - Paris 6</orgName><orgName type="acronym">UPMC</orgName><desc><address><addrLine>4 place Jussieu - 75005 Paris</addrLine><country key="FR"></country></address><ref type="url">http://www.upmc.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Sagot, Benoit" sort="Sagot, Benoit" uniqKey="Sagot B" first="Benoît" last="Sagot">Benoît Sagot</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-54505" status="OLD"><idno type="RNSR">200818336A</idno><orgName>Analyse Linguistique Profonde à Grande Echelle ; Large-scale deep linguistic processing</orgName><orgName type="acronym">ALPAGE</orgName><date type="end">2016-01-31</date><desc><address><addrLine>Université Paris Diderot, Bât. Olympe de Gouges, case postale 7003, 75205 Paris cedex 13 - INRIA Rocquencour</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/alpage</ref></desc><listRelation><relation active="#struct-86790" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-300301" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-86790" type="direct"><org type="laboratory" xml:id="struct-86790" status="VALID"><idno type="RNSR">196718247G</idno><orgName>INRIA Paris-Rocquencourt</orgName><desc><address><addrLine>INRIA Rocquencourt : Domaine de Voluceau, Rocquencourt B.P. 105 78153 le Chesnay Cedex / INRIA Paris - 23 avenue d'Italie 75013 Paris</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/centre/paris-rocquencourt</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-300301" type="direct"><org type="institution" xml:id="struct-300301" status="VALID"><orgName>Université Paris Diderot - Paris 7</orgName><orgName type="acronym">UP7</orgName><desc><address><addrLine>5 rue Thomas-Mann - 75205 Paris cedex 13</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-paris-diderot.fr</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Mariani, Joseph" sort="Mariani, Joseph" uniqKey="Mariani J" first="Joseph" last="Mariani">Joseph Mariani</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-202" status="OLD"><orgName>Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur [Orsay]</orgName><orgName type="acronym">LIMSI</orgName><desc><address><addrLine>Université Paris Sud (Paris XI) Bât. 508 BP 133 91403 ORSAY CEDEX</addrLine><country key="FR"></country></address><ref type="url">http://www.limsi.fr/</ref></desc><listRelation><relation name="UPR3251" active="#struct-441569" type="direct"></relation><relation active="#struct-92966" type="direct"></relation><relation active="#struct-93591" type="direct"></relation></listRelation><tutelles><tutelle name="UPR3251" active="#struct-441569" type="direct"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-92966" type="direct"><org type="institution" xml:id="struct-92966" status="VALID"><orgName>Université Paris-Sud - Paris 11</orgName><orgName type="acronym">UP11</orgName><desc><address><addrLine>Bâtiment 300 - 91405 Orsay cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.u-psud.fr/</ref></desc></org></tutelle><tutelle active="#struct-93591" type="direct"><org type="institution" xml:id="struct-93591" status="VALID"><orgName>Université Pierre et Marie Curie - Paris 6</orgName><orgName type="acronym">UPMC</orgName><desc><address><addrLine>4 place Jussieu - 75005 Paris</addrLine><country key="FR"></country></address><ref type="url">http://www.upmc.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country></affiliation></author><author><name sortKey="Couillault, Alain" sort="Couillault, Alain" uniqKey="Couillault A" first="Alain" last="Couillault">Alain Couillault</name><affiliation wicri:level="1"><hal:affiliation type="laboratory" xml:id="struct-40831" status="VALID"><orgName>Laboratoire Informatique, Image et Interaction</orgName><orgName type="acronym">L3I</orgName><desc><address><addrLine>Bâtiment Pascal Avenue Michel Crépeau F-17042 La Rochelle Cedex 1</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lr.fr/l3i</ref></desc><listRelation><relation name="EA2118" active="#struct-300311" type="direct"></relation></listRelation><tutelles><tutelle name="EA2118" active="#struct-300311" type="direct"><org type="institution" xml:id="struct-300311" status="VALID"><orgName>Université de La Rochelle</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">La Rochelle</settlement><region type="region" nuts="2">Nouvelle-Aquitaine</region><region type="old region" nuts="2">Poitou-Charentes</region></placeName><orgName type="university">Université de La Rochelle</orgName></affiliation></author></analytic><idno type="DOI">10.1007/978-3-319-08958-4_25</idno></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="mix" xml:lang="en"><term>Amazon Mechanical Turk</term><term>Ethics</term><term>Language resources</term></keywords><keywords scheme="Wicri" type="topic" xml:lang="fr"><term>éthique</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">This article is a position paper about Amazon Mechanical Turk, the use of which has been steadily growing in language processing in the past few years. According to the mainstream opinion expressed in articles of the domain, this type of on-line working platforms allows to develop quickly all sorts of quality language resources, at a very low price, by people doing that as a hobby. We shall demonstrate here that the situation is far from being that ideal. Our goal here is manifold: 1- to inform researchers, so that they can make their own choices, 2- to develop alternatives with the help of funding agencies and scientific associations, 3- to propose practical and organizational solutions in order to improve language resources development, while limiting the risks of ethical and legal issues without letting go price or quality, 4- to introduce an Ethics and Big Data Charter for the documentation of language resource</div></front></TEI><affiliations><list><country><li>France</li></country><region><li>Grand Est</li><li>Lorraine (région)</li><li>Nouvelle-Aquitaine</li><li>Poitou-Charentes</li></region><settlement><li>La Rochelle</li><li>Metz</li><li>Nancy</li></settlement><orgName><li>Université de La Rochelle</li><li>Université de Lorraine</li></orgName></list><tree><country name="France"><region name="Grand Est"><name sortKey="Fort, Karen" sort="Fort, Karen" uniqKey="Fort K" first="Karen" last="Fort">Karen Fort</name></region><name sortKey="Adda, Gilles" sort="Adda, Gilles" uniqKey="Adda G" first="Gilles" last="Adda">Gilles Adda</name><name sortKey="Couillault, Alain" sort="Couillault, Alain" uniqKey="Couillault A" first="Alain" last="Couillault">Alain Couillault</name><name sortKey="Mariani, Joseph" sort="Mariani, Joseph" uniqKey="Mariani J" first="Joseph" last="Mariani">Joseph Mariani</name><name sortKey="Sagot, Benoit" sort="Sagot, Benoit" uniqKey="Sagot B" first="Benoît" last="Sagot">Benoît Sagot</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000A18 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000A18 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Main
|étape= Exploration
|type= RBID
|clé= Hal:hal-01053047
|texte= Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use
}}
| This area was generated with Dilib version V0.6.33. |  |